Three dimensional (3D) video has gained significant interest recently. Along with the advances in capture and display technologies, 3D video is becoming a reality in consumer domain with different application opportunities. 3D video applications can be grouped under three categories, free-viewpoint video, 3D TV (video), and immersive teleconferencing. The requirements of these applications are quite different and each category has its own challenges.
One technology fostering the feasibility of various envisioned 3D video applications is the multi-view video coding (MVC) techniques. In multi-view video coding, video sequences output from different cameras, each corresponding to a different view, are encoded into one bit-stream. At the client side, a particular view can be reconstructed and displayed by decoding the pictures belonging to that view. It is also possible that more than one view is reconstructed and displayed.
When transmitting 3D content based on 2D images, the bandwidth constrain is an important issue, thus a powerful compressor is required to code the 3D content with only reasonable number of views. However, at the client, the user may wish to view the 3D content at any angle. Therefore, it is desirable that the decoder is able to render views as much as possible and as continuous as possible. View synthesis can be used to solve the problem by transmitting reasonable number of views while interpolating other views at the renderer.
However, the prior art view synthesis arrangements have some technical shortcomings, especially relating to warped depth images. For example, there has not been presented an efficient method, suitable for 3D video applications, for tackling point-like artifacts, such as occlusion artifacts and pinholes, in the images, or visibility problems when multiple pixels from reference images are warped into the same pixel of a destination image.